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DECODING APPARATUS AND METHOD, AND STORAGE 
MEDIUM STORING DECODING PROCESSING PROGRAM OF THE SAME 



m 



BACKGROUND OF THE INVENTION 
5 Field of the Invention 

The present invention relates to a data decoding 
apparatus and method, and a storage medium and, more 
particularly, to decoding processing of decoding a 
..^ bitstream such as coded and multiplexed image and audio 

10 signals in a digital TV reception apparatus or a 
T-i digital storage medium reproduction apparatus. 

Related Background Art 
3- Fig. 1 shows a conventional decoding apparatus for 

3 decoding a coded bitstream. Fig. 1 shows the 

15 arrangement of a digital broadcasting receiver that has 

3 an accounting function. Referring to Fig. 1, the 

3 

^ apparatus comprises a bitstream input terminal 101 for 

receiving a coded bitstream, descrambling circuit 102, 
and separation circuit 103 for separating multiplexed 
20 data. 

The apparatus also has an audio signal decoding 
circuit 104 for decoding a separated audio signal, a 
video signal decoding circuit 105 for decoding a 
separated video signal, a CPU 106, an IC card 107, and 
25 a remote controller 108. The operation of the 

conventional digital broadcasting receiver will be 
described below with reference to Fig. 1. 
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A bitstream received by a tuner (not; shown) in 
Fig- 1 is input from the input terminal 101 to the 
apparatus. As the coding scheme, so-called MPEG 
(Moving Picture Experts Group) 2 is used. 
5 In this input stream, normally, a coded audio 

signal and video signal are multiplexed and scrambled 
to permit only a specific user to view the contents. 

A method of confirming viewing permission will be 
described later. If viewing is permitted, the stream 
10 input to the descrambling circuit 102 is descrambled by 
O a descrambling key given by the CPU 106 and input to 

^rJ the separation circuit 103. 

The separation circuit 103 separates the stream 
O into an audio signal stream and a video signal stream, 

15 which are supplied to the decoding circuits 104 and 
;=3 105, respectively, decoded, and displayed on a monitor 

(not shown). Control lines from the CPU 106 are 
connected to the audio signal decoding circuit 104 and 
video signal decoding circuit 105 to control 
20 stereo/multi-lingual data or display information on the 
screen. 

A method of confirming viewing permission 
(authentication method) will be described. 

A subscribed program ID and descrambling key are 
25 stored in the IC card 107 in advance. To watch the 
program, the CPU 106 accesses the IC card 107 and 
confirms viewing permission on the basis of whether the 
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program ID stored ±n the IC card and the program of the 
input stream coincide with each other. In this case, 
programs that are not subscribed in advance cannot be 
watched . 

5 Conventionally, however, in any schemes, viewing 

is limited in units of channels or in units of 
programs. In addition, only ON/OFF-control can be 
performed. For this reason, it is difficult to provide 
higher-quality video data in accordance with 
^ 10 subscription or provide an interactive program. 

□ In recent years, large-capacity media such as DVD 

H are becoming popular, and not only fixed control for 

reproduction/non-reproduction but also a variety of 

Q flexible use forms that take advantage of the large 

J- 

i,:^ 15 capacity are required. However, such a service is hard 

I. J: 

S to realize. 

SUMMARY OF THE INVENTION 

The present invention has been made in 

20 consideration of the aibove situation, and has as its 

object to provide a decoding apparatus and method and a 
storage medium storing. a decoding processing program, 
which permit to set a multi-level reproduction 
processing of an object when bitstream which is coded 

25 and multiplexed in units of a plurality of objects is 
decoded . 
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In order to achieve the^ above object:, according to 
an aspect of the present invention, there is provided a 
decoding apparatus/method in which a bitstream obtained 
by coding a plurality of object data in units of 
5 objects and multiplexing the coded data is input, coded 
data of each object is separated from the bitstream, a 
predetermined object is selected from the plurality of 
objects contained in the bitstream, the coded data of 
^ the selected object is decoded and object data is 

=0. 10 output, and the decoded object data is synthesized. 

m 

^ According to another aspect of the present 

=n invention, there is provided a computer-readable 

■ 

storage medium which stores a decoding program 
Q comprising input processing of inputting a bitstream 

: %^ 

15 obtained by coding a plurality of object data in units 
1*5 of objects and multiplexing the coded data, separation 

processing of separating coded data of each object from 
the bitstream, selection processing of selecting a 
predetermined object from the plurality of objects 
20 contained in the bitstream, decoding processing of 
decoding the coded data of the selected object and 
outputting object data, and synthesis processing of 
synthesizing the decoded object data. 

Other objects, features and advantages of the 
25 invention become apparent from the following detailed 
description taken in conjunction with the accompanying 
drawings . 
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BRIEF DESCRIPTION OF THE DRAWINGS 

Fig. 1 is a block diagram showing -bhe arrangement: 
of a conven-tlonal decoding apparatus; 

Fig. 2 Is a block diagram showing the arrangement 
of a decoding apparatus according to an embodiment; 

Figs. 3A, 3B, 3C and 3D are views for explaining 
the operation of a priority decoder; and 

Fig. 4 Is a block diagram showing details of a 
syntheslzatlon circuit 210 shown In Fig. 1. 

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENT 

In recent years, MPEG4 has received a great deal 
of attention as a new coding scheme. As some 
characteristic features of MPEG4, it has scalability in 
space and time, and additionally, it divides video data 
into a plurality of objects (e.g., background and 
persons), and codes and multiplexes respective data, 
unlike the conventional MPEG2. 

As some other characteristic features,, coding and 
multiplex corresponding to CG or animation are 
possible, and the error resilience is high. This 
embodiment places an emphasis on the processing of a 
plurality of objects. In this embodiment, an MPEG4 
bitstream is decoded. 

Fig, 2 is a block diagram showing the arrangement 
of a decoding apparatus according to this embodiment . 



Referring to Fig. 2, the decoding apparatus has an 
antenna 201 for receiving a broadcasting signal, and a 
tuner 202 for demodulating the reception signal 
received by the antenna 201 to obtain a coded 
bitstream. 

The apparatus also has a descrambling circuit 203, 
a separation circuit 204 for separating multiplexed 
data, and an audio signal decoding circuit 205 for 
decoding a separated audio signal. 

Decoding circuits 206 to 208 decode separated VOP 
(Video Object Plane) signals, respectively. A scene 
description decoding circuit 209 decodes a scene 
description. A synthesization circuit 210 synthesizes 
the audio signal and VOP signals. 

The decoding apparatus also has switches 211 to 
213, a priority decoder 214, a communication circuit 
215 for connecting the apparatus to a communication 
line such as a telephone line or a line of the Internet 
or cable TV, a CPU 216, an IC card 217, a remote 
controller 218, and a display apparatus 219 for audio 
and image output. 

The processing operation of the decoding apparatus 
of this embodiment having the above-described 
arrangement will be described below. 

An iyiPEG4 bitstream obtained through the antenna 
201 and tuner 202 or an MPEG4 bitstream obtained 
through the communication circuit 215 is selectively 



input -to the descrambling circuit 203. If viewing is 
permitted, a descrambling key is supplied from the CPU 
216, and the descrambling circuit 203 descrambles the 
bitstream. 

If viewing is not permitted, video data for 
notifying the user that viewing is not permitted is 
generated by the CPU 216 and displayed on the display 
apparatus 219 through the synthesization circuit 210. 

A method of authenticating viewing permission will 
be described below. 

First, the user subscribes to a management center 
(a company that supplies bitstreams or a management 
company entrusted by the company) through the 
communication circuit 215 in advance, downloads a 
program ID and descrambling key from the management 
center, and stores them in the IC card 217. 

To manage and protect intellectual properties such 
as a copyright contained in the MPEG4 bitstream, the 
bitstream contains IPMP ( Intellectual Property and 
Management Protection) data. The IPMP data contains 
the same program ID as that stored in the IC card 217. 

The CPU 216 extracts the IPMP data from the 
bitstream input to the descrambling circuit 203 and 
detects the program ID. 

Next, the CPU 216 reads out the program ID stored 
in the IC card 217 and authenticates whether the 
readout program ID coincides with the program ID 
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detected from the IPMP data. If the two program IDs 
coincide with each other, it is determined that viewing 
is permitted. In this case, the program cannot be 
watched unless the user subscribes in advance. 
5 In addition to the above method, a method that 

charges for viewing can also be used to obtain viewing 
permission. 

In this case, a descrambling key used for 
descrambling and communication information (e.g., a 

10 URL) for contact with the management center are 

contained in the IPMP data. The viewer can descramble 
the data using the descrambling key contained in the 
IPMP data at any time. When the descrambling key is 
used, personal information and viewing record stored in 

15 the IC card 217 are transmitted to the management 

center on the basis of the communication information, 
and charges incurred are billed later. That is, a 
pay-per-view scheme is employed . 

In this embodiment, a viewing age limiting 

20 function is prepared. Unless a password is input 

through the remote controller 218, a specific program 

(e.g., an adult program) cannot be watched 

( descrambled ) even if charges have been payed. 

Processing after viewing is permitted and the data 

25 is descrambled by the above-described method will be 
described next . 



The descr ambled st:ream is separated by "the 
separation circuit 204 into streams such as audio, 
VOPO, VOPl, VOP2, and scene description streams, VOP 
is the basic unit of a video image in MPEG4 and 
actually represents the instantaneous value (image) of 
a video object. 

MPEG4 has a number of layers including an upper 
layer called a VOL (Video Object Layer) in which, e.g., 
a VOP is expanded in space or time to increase 
resolution, and a VO (Video Object) that is a 
collection of VOLs. In this embodiment, a basic VOP 
will be exemplified for the descriptive convenience. 
The number of VOPs is limited to three in this example. 

The streams separated by the separation circuit 
204 are decoded by the audio signal decoding circuit 
205, VOP decoding circuits 206, 207, and 208, and scene 
description decoding circuit 209, synthesized by the 
synthesization circuit 210, and output to the display 
apparatus 219. 

The scene description is a description using a 
language, which positions VOPs divided into objects in 
space and synchronizes them in time to obtain one video 
image (audio data is also regarded as one object and 
synchronized in time). 

The reason why not only spatial positioning but 
also temporal synchronization is necessary is that each 
object is influenced by time shift due to multiplex or 
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variations due to decoding delay. Actually, buffers 
must be prepared on the input and outputs sides of each 
decoding circuit in Fig. 2, although they are omitted 
in Fig. 2 for the illustrative convenience. 
5 Referring back to Fig. 2, the switches 211, 212, 

and 213 determine whether VOP decoding outputs are to 
be output to the synthesization circuit 210. When 
these switches are turned off, an output different from 
the original MPEG4 video output is obtained. 

10 More specifically, when a certain switch is turned 

off, an object corresponding to the switch is not 
displayed on the screen. The switches 211, 212, and 
213 are ON/OFF-controlled by the priority decoder 214. 
The priority decoder 214 functions as shown in 

15 Figs. 3A to 3D. That is, the priority decoder 214 

ON/OFF-controls the switches 211 to 213 in accordance 
with a value output from the CPU 216. When priority 
decoder VOPO, 1, 2, . • . are assigned sequentially from 
an important video object in advance before coding 

20 video data, an integer value 0, 1, 2, or 3 is supplied 
in accordance with the subscription of the user, 
thereby providing a service by preparing a multiple of 
resolutions to one program. 

As the subscription procedures, only one extra 

25 integer value for object selection is stored in the IC 
card in addition to the program ID and descrambling 
key. In pay-per-view as well, the numerical value need 



only be added to viewing information to be transmitted 
to the management center . 

An explicit example of discrimination of the 
service will be described with reference to the block 
diagram of Fig. 4. Fig. 4 shows a specific example of 
the synthesization circuit 210 shown in Fig. 2. A VOP 
in Fig. 4 corresponds to a VOL in which the VOP is 
expanded to increase resolution. VOPO is an image 
having the lowest resolution. If VOPl and V0P2 are not 
given, an image obtained by simply expanding a 
subwindow in the vertical and horizontal directions is 
obtained as a video output. 

VOPl or V0P2 is high-resolution image information. 
When VOPl or V0P2 is supplied to the synthesization 
circuit 210, an image with higher resolution is 
obtained as a video output. Hence, images of a 
plurality of image quality levels can be provided in 
units of subscription contents of users in 
correspondence with a common bitstream in accordance 
with integer values according to the subscriptions. 

The numerical values shown in Figs. 3A to 3D will 
be exemplified. A user who has subscribed to "3" in 
Fig. 3D receives a service with the highest image 
quality. Referring to only Figs. 3A to 3D, it appears 
this system can be implemented using MPEG2 hierarchical 
coding. However, herein the basic concept is "video 
object", and the resolution of only an object (part) on 
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the window can be changed. MPEG2 permits only 
hierarchical coding targeting the entire picture. 

A supplementary explanation will be done below in 
association with this embodiment. In this embodiment, 
the priority of VOPs is determined using integer 
values. Instead, command mnemonic or the like may be 
used. Alternatively, the current level may be stored, 
and the level may be changed by an UP/DOWN command. 

In this embodiment, the integer values coincide 
with the number of VOPs to be decoded. Since the 
number of VOPs always varies, a ratio to the maximum 
allowable number of VOPs (e.g., 50% decoding or 20% 
decoding) may be used as an index. 

Referring to Fig. 2, the switches 211, 212, and 
213 are controlled to determine the priority of the 
VOPs- However, the VOP decoding circuits 206, 207, and 
208 may be controlled to inhibit that decoding is 
performed selectively . 

Instead of using the priority decoder 214, direct 
control may be done by the CPU 216 (decoding by 
software). In this embodiment, audio objects have not 
been mentioned. For example, for an application in 
which a plurality of audio data are multiplexed in a 
one-to-one correspondence with VOPs, processing for 
VOPs in this embodiment can be applied. 

When this embodiment is applied while multiplexing 
a plurality of scene descriptions, a service with a 
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more flexible hierarchical structure can be provided as 
compared to this embodiment in which priority is given 
to VOPs. In this embodiment, the description has been 
made assuming broadcasting. However, this embodiment 
5 can also be applied to package media such as a DVD. 
This embodiment can also be applied to an 
interactive (bidirectional) application such as a quiz 
program because ON/OFF control and other control can be 
performed in units of objects. 

:«=^ 

=3 10 As described above, according to this embodiment, 

the precision when a bitstream coded and multiplexed in 
\fl units of a plurality of objects is separated into 

Q object streams and decoded can be controlled to a 

rj plurality of levels. For this reason, a variety of 

15 flexible services can be realized for an input 

bitstream. For example, in a broadcasting medium, 
viewing can be conventionally limited in units of 
channels or in units of programs. In addition, only 
ON/OFF control can be performed. However, a 
20 high-quality image can be provided in accordance with 
the subscription even for one program. 

In addition, additional circuits or additional 
software for priority assignment can be minimized. 
When priority is given to VOPs on the coding side, any 
25 large failure can be avoided even in an apparatus 

having a low ability and capable of decoding only two 
VOPs on the receiving side. 
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The present inven-tion may be applied -to a system 
const:i-tu1:ed by a plurality of devices (e.g,, a host 
computer or an interface device) or an apparatus 
comprising a single device. 

The present invention also incorporates an 
arrangement in which various devices are operated such 
that the function of the embodiment is realized by 
supplying software program codes for realizing the 
function of the above embodiment to the computer in the 
apparatus or system connected to the various devices, 
and causing the various devices to operate in 
accordance with the program stored in the computer (CPU 
or MPU) of the system or apparatus. 

In this case, the software program codes realize 
the function of the above-described embodiment by 
themselves, and the program codes or means for 
supplying the program codes to the computer, e.g., a 
storage medium storing the program codes constitute the 
present invention. As a storage medium storing the 
program codes, a floppy disk, a hard disk, an optical 
disk, a magnetooptical disk, a CD-ROM, a magnetic tape, 
a nonvolatile memory card, a ROM, or the like can be 
used . 

The function of the above-described embodiment is 
realized not only when the supplied program codes are 
executed by the computer but also when the OS 
(Operating System) running on the computer realizes the 
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function of trhe above embodiment: in collaboration with 
those program codes . 

The present invention also incorporates an 
arrangement in which the function of the 
above-described embodiments is realized when the 
supplied program codes are written in the memory of a 
function expansion board inserted into the computer or 
a function expansion unit connected to the computer, 
and the CPU of the function expansion board or function 
expansion unit performs part or all of actual 
processing on the basis of the instructions of the 
program codes • 

In other words, the foregoing description of 
embodiments has been given for illustrative purposes 
only and not to be constructed as imposing any 
limitation in every respect. 

The scope of the invention is, therefore, to be 
determined solely by the following claims and not 
limited by the text of the specifications and 
alternations made within a scope equivalent to the 
scope of the claims fall within the true spirit and 
scope of the invention. 



